Learning with Whom to Communicate Using Relational Reinforcement Learning.
Relational reinforcement learning (RRL) has emerged in the machine learning community as a new promising subfield of reinforcement learning (RL) (e.g. [1]). It upgrades RL techniques by using relational representations for states, actions and learned valuefunctions
Frequent subgraph mining in outerplanar graphs.
In recent years there has been an increased interest in frequent pattern discovery in large databases of graph structured objects. While the frequent connected subgraph mining problem for tree datasets can be solved in incremental polynomial time, it becomes intractable for arbitrary graph databases. Existing approaches have therefore resorted to various heuristic strategies and restrictions of the search space, but have not identified a practically relevant tractable graph class beyond trees. In this paper, we consider the class of outerplanar graphs, a strict generalization of trees, develop a frequent subgraph mining algorithm for outerplanar graphs, and show that it works in incremental polynomial time for the practically relevant subclass of well-behaved outerplanar graphs, i.e., which have only polynomially many simple cycles. We evaluate the algorithm empirically on chemo- and bioinformatics applications. 1
Prediction of Clinical Conditions after Coronary Bypass Surgery using Dynamic Data Analysis.

A comparison of pruning criteria for probability trees.

Efficient frequent connected subgraph mining in graphs of bounded tree%2Dwidth.

Efficient frequent connected subgraph mining in graphs of bounded tree%2Dwidth.

Proceedings of the ACM SIGKDD Workshop on Statistical and Relational Learning in Bioinformatics

Learning with whom to communicate using relational reinforcement learning.
Relational reinforcement learning (RRL) has emerged in the machine learning community as a new promising subfield of reinforcement learning (RL) (e.g. [1]). It upgrades RL techniques by using relational representations for states, actions and learned valuefunctions
Monte%2DCarlo Tree Search in Poker Using Expected Reward Distributions.
Poker playing computer bots can be divided into two categories. There are the game-theoretic bots, that play according to a strategy that gives rise to a Nash equilibrium. These bots are impossible to beat, but are also not able to exploit non-optimalities in their opponents. The other type of bot is the exploiting bot that employs game tree search and opponent modeling techniques to discover and exploit weaknesses of
Handling missing values and censored data in PCA of pharmacological matrices.

Dynamic Data Analysis and Data Mining for Prediction of Clinical Stability.

Polynomial%2DDelay Enumeration of Monotonic Graph Classes.

Deriving distance metrics from generality relations.

StReBio%2709 statistical relational learning and mining in bioinformatics.
Bioinformatics is an application domain where information is naturally represented in terms of relations between heterogenous objects. Modern experimentation and data acquisition techniques allow the study of complex interactions in biological systems. This raises interesting challenges for machine learning and data mining researchers, as the amount of data is huge, some information can not be observed, and measurements may be noisy. This report presents a review on the ACM SIGKDD 2009 Workshop on Statistical Relational Learning and Mining in Bioinformatics (StReBio’09) which was held in Paris on June 28th, 2009. The aim of this workshop was to provide a forum to share challenges, results and ideas at the frontier between the field of statistical relational learning and the field of bioinformatics. 1.


Bayes%2DRelational Learning of Opponent Models from Incomplete Information in No%2DLimit Poker.
We propose an opponent modeling approach for nolimit Texas hold-em poker that starts from a (learned) prior, i.e., general expectations about opponent behavior and learns a relational regression tree-function that adapts these priors to specific opponents. An important asset is that this approach can learn from incomplete information (i.e. without knowing all players ’ hands in training games).
Active Learning for High Throughput Screening.

An Efficiently Computable Graph%2DBased Metric for the Classification of Small Molecules.

Using Decision Trees as the Answer Networks in Temporal Difference%2DNetworks.
State representation for intelligent agents is a continuous challenge as the need for abstraction is unavoidable in large state spaces. Predictive representations offer one way to obtain state abstraction by replacing a state with a set of predictions about future interactions
Efficient Frequent Connected Subgraph Mining in Graphs of Bounded Treewidth.

Anti%2Dmonotonic Overlap%2DGraph Support Measures.

Learning directed probabilistic logical models ordering%2Dsearch versus structure%2Dsearch.
Abstract. We discuss how to learn non-recursive directed probabilistic logical models from relational data. This problem has been tackled before by upgrading the structure-search algorithm initially proposed for Bayesian networks. In this paper we propose to upgrade another algorithm, namely ordering-search, since for Bayesian networks this was found to work better than structure-search. We experimentally compare the two upgraded algorithms on two relational domains. We conclude that there is no significant difference between the two algorithms in terms of quality of the learnt models while ordering-search is significantly faster.
Generalized ordering%2Dsearch for learning directed probabilistic logical models.
Abstract. Recently, there has been an increasing interest in directed probabilistic logical models and a variety of languages for describing such models has been proposed. Although many authors provide high-level arguments to show that in principle models in their language can be learned from data, most of the proposed learning algorithms have not yet been studied in detail. We introduce an algorithm, generalized ordering-search, to learn both structure and conditional probability distributions (CPDs) of directed probabilistic logical models. The algorithm upgrades the ordering-search algorithm for Bayesian networks. We use relational probability trees as a representation for the CPDs. We present experiments on blocks world domains, a gene domain and the Cora dataset. 1
On Policy Learning in Restricted Policy Spaces.

Learning Directed Probabilistic Logical Models Ordering%2DSearch Versus Structure%2DSearch.
Abstract. We discuss how to learn non-recursive directed probabilistic logical models from relational data. This problem has been tackled before by upgrading the structure-search algorithm initially proposed for Bayesian networks. In this paper we propose to upgrade another algorithm, namely ordering-search, since for Bayesian networks this was found to work better than structure-search. We experimentally compare the two upgraded algorithms on two relational domains. We conclude that there is no significant difference between the two algorithms in terms of quality of the learnt models while ordering-search is significantly faster.
Transfer Learning in Reinforcement Learning Problems Through Partial Policy Recycling.
Abstract. In this paper we investigate the relation between transfer learning in reinforcement learning with function approximation and supervised learning with concept drift. We present a new incremental relational regression tree algorithm that is capable of dealing with concept drift through tree restructuring and show that it enables a reinforcement learner, more precisely a Q-learner, to transfer knowledge from one task to another by recycling those parts of the generalized Q-function that still hold interesting information for the new task. We illustrate the performance of the algorithm in experiments with both supervised learning tasks with concept drift and reinforcement learning tasks that allow the transfer of knowledge from easier, related tasks.
Online Learning and Exploiting Relational Models in Reinforcement Learning.
In recent years, there has been a growing interest in using rich representations such as relational languages for reinforcement learning. However, while expressive languages have many advantages in terms of generalization and reasoning, extending existing approaches to such a relational setting is a non-trivial problem. In this paper, we present a first step towards the online learning and exploitation of relational models. We propose a representation for the transition and reward function that can be learned online and present a method that exploits these models by augmenting Relational Reinforcement Learning algorithms with planning techniques. The benefits and robustness of our approach are evaluated experimentally. 1
Learning Directed Probabilistic Logical Models Using Ordering%2DSearch.


A Polynomial%2Dtime Metric for Outerplanar Graphs.




Generalized Ordering%2DSearch for Learning Directed Probabilistic Logical Models.
Abstract. Recently, there has been an increasing interest in directed probabilistic logical models and a variety of languages for describing such models has been proposed. Although many authors provide high-level arguments to show that in principle models in their language can be learned from data, most of the proposed learning algorithms have not yet been studied in detail. We introduce an algorithm, generalized ordering-search, to learn both structure and conditional probability distributions (CPDs) of directed probabilistic logical models. The algorithm upgrades the ordering-search algorithm for Bayesian networks. We use relational probability trees as a representation for the CPDs. We present experiments on blocks world domains, a gene domain and the Cora dataset. 1
ReMauve A Relational Model Tree Learner.

Frequent subgraph mining in outerplanar graphs.
In recent years there has been an increased interest in frequent pattern discovery in large databases of graph structured objects. While the frequent connected subgraph mining problem for tree datasets can be solved in incremental polynomial time, it becomes intractable for arbitrary graph databases. Existing approaches have therefore resorted to various heuristic strategies and restrictions of the search space, but have not identified a practically relevant tractable graph class beyond trees. In this paper, we consider the class of outerplanar graphs, a strict generalization of trees, develop a frequent subgraph mining algorithm for outerplanar graphs, and show that it works in incremental polynomial time for the practically relevant subclass of well-behaved outerplanar graphs, i.e., which have only polynomially many simple cycles. We evaluate the algorithm empirically on chemo- and bioinformatics applications. 1
Frequent Subgraph Mining in Outerplanar Graphs.
In recent years there has been an increased interest in frequent pattern discovery in large databases of graph structured objects. While the frequent connected subgraph mining problem for tree datasets can be solved in incremental polynomial time, it becomes intractable for arbitrary graph databases. Existing approaches have therefore resorted to various heuristic strategies and restrictions of the search space, but have not identified a practically relevant tractable graph class beyond trees. In this paper, we consider the class of outerplanar graphs, a strict generalization of trees, develop a frequent subgraph mining algorithm for outerplanar graphs, and show that it works in incremental polynomial time for the practically relevant subclass of well-behaved outerplanar graphs, i.e., which have only polynomially many simple cycles. We evaluate the algorithm empirically on chemo- and bioinformatics applications. 1
Refining Aggregate Conditions in Relational Learning.
In relational learning, predictions for an individual are based not only on its own properties but also on the properties of a set of related individuals. Many systems use aggregates to summarize this set. Features thus introduced compare the result of an aggregate function to a threshold. We consider the case where the set to be aggregated is generated by a complex query and present a framework for refining such complex aggregate conditions along three dimensions: the aggregate function, the query used to generate the set, and the threshold value. The proposed aggregate refinement operator allows a more efficient search through the hypothesis space and thus can be beneficial for many relational learners that use aggregates. As an example application, we have implemented the refinement operator in a relational decision tree induction system. Experimental results show a significant efficiency gain in comparison with the use of a less advanced refinement operator. 
Graph kernels and Gaussian processes for relational reinforcement learning.
Relational reinforcement learning is a Q-learning technique for relational state-action spaces. It aims to enable agents to learn how to act in an environment that has no natural representation as a tuple of constants. In this case, the learning algorithm used to approximate the mapping between state-action pairs and their so called Q(uality)-value has to be not only very reliable, but it also has to be able to handle the relational representation of state-action pairs. In this paper we investigate...
Logical Bayesian Networks and Their Relation to Other Probabilistic Logical Models.
We review Logical Bayesian Networks, a language for probabilistic logical modelling, and discuss its relation to Probabilistic Relational Models and Bayesian Logic Programs. 1 Probabilistic Logical Models Probabilistic logical models are models combining aspects of probability theory with aspects of Logic Programming, first-order logic or relational languages. Recently a variety of languages to describe such models has been introduced. For some languages techniques exist to learn such models from data. Two examples are Probabilistic Relational Models (PRMs) [4] and Bayesian Logic Programs (BLPs) [5]. These two languages are probably the most popular and well-known in the Relational Data Mining community. We introduce a new language, Logical Bayesian Networks (LBNs) [2], that is strongly related to PRMs and BLPs yet solves some of their problems with respect to knowledge representation (related to expressiveness and intuitiveness). PRMs, BLPs and LBNs all follow the principle of Knowledge Based Model Construction: they offer a language that can be used to specify general probabilistic logical knowledge and they provide a methodology to construct a propositional model based on this knowledge when given a specific
A Comparison of Approaches for Learning Probability Trees.
Probability trees (or Probability Estimation Trees, PET&#039;s)  are decision trees with probability distributions in the leaves. Several  alternative approaches for learning probability trees have been proposed  but no thorough comparison of these approaches exists.
Logical Bayesian Networks and Their Relation to Other Probabilistic Logical Models.
We review Logical Bayesian Networks, a language for probabilistic logical modelling, and discuss its relation to Probabilistic Relational Models and Bayesian Logic Programs. 1 Probabilistic Logical Models Probabilistic logical models are models combining aspects of probability theory with aspects of Logic Programming, first-order logic or relational languages. Recently a variety of languages to describe such models has been introduced. For some languages techniques exist to learn such models from data. Two examples are Probabilistic Relational Models (PRMs) [4] and Bayesian Logic Programs (BLPs) [5]. These two languages are probably the most popular and well-known in the Relational Data Mining community. We introduce a new language, Logical Bayesian Networks (LBNs) [2], that is strongly related to PRMs and BLPs yet solves some of their problems with respect to knowledge representation (related to expressiveness and intuitiveness). PRMs, BLPs and LBNs all follow the principle of Knowledge Based Model Construction: they offer a language that can be used to specify general probabilistic logical knowledge and they provide a methodology to construct a propositional model based on this knowledge when given a specific
Multi%2Dagent Relational Reinforcement Learning.
Abstract. In this paper we study Relational Reinforcement Learning in a multi-agent setting. There is growing evidence in the Reinforcement Learning research community that a relational representation of the state space has many benefits over a propositional one. Complex tasks as planning or information retrieval on the web can be represented more naturally in relational form. Yet, this relational structure has not been exploited for multi-agent reinforcement learning tasks and has only been studied in a single agent context so far. This paper is a first attempt in bridging the gap between Relation Reinforcement Learning (RRL) and Multi-agent Systems (MAS). More precisely, we will explore how a relational structure of the state space can be used in a Multi-Agent Reinforcement Learning context. 1
Searching for Compound Goals Using Relevancy Zones in the Game of Go.

Condensed Representations for Inductive Logic Programming.
When mining frequent Datalog queries, many queries will cover the same examples; i.e., they will be equivalent and hence, redundant. The equivalences can be due to the data set or to the regularities specified in the background theory. To avoid the generation of redundant clauses, we introduce various types of condensed representations. More specifically, we introduce δ-free and closed clauses, that are defined w.r.t. the data set, and semantically free and closed clauses, that take into account a logical background theory. A novel algorithm that employs these representations is also presented and experimentally evaluated on a number of benchmark problems in inductive logic programming.
Compact Representation of Knowledge Bases in Inductive Logic Programming.
In many applications of Inductive Logic Programming (ILP), learning occurs from a knowledge base that contains a large number of examples. Storing such a knowledge base...
Relational Instance Based Regression for Relational Reinforcement Learning.
The full paper on this topic appears in the Proceedings of the Twentieth International
Graph Kernels and Gaussian Processes for Relational Reinforcement Learning.
Relational reinforcement learning is a Q-learning technique for relational state-action spaces. It aims to enable agents to learn how to act in an environment that has no natural representation as a tuple of constants. In this case, the learning algorithm used to approximate the mapping between state-action pairs and their so called Q(uality)-value has to be not only very reliable, but it also has to be able to handle the relational representation of state-action pairs. In this paper we investigate...
Compact Representation of Knowledge Bases in ILP.
Many inductive systems, including ILP systems, learn from  a knowledge base that is structured around examples. In practical situations  this example-centered representation can cause a lot of redundancy.
Clustering and instance based learning in first order logic.

Improving the Efficiency of Inductive Logic Programming Through the Use of Query Packs.
Inductive logic programming, or relational learning, is a powerful paradigm for machine learning or data mining. However, in order for ILP to become practically useful, the efficiency of ILP systems must improve substantially. To this end, the notion of a query pack is introduced: it structures sets of similar queries. Furthermore, a mechanism is described for executing such query packs. A complexity analysis shows that considerable efficiency improvements can be achieved through the use of this query pack execution mechanism. This claim is supported by empirical results obtained by incorporating support for query pack execution in two existing learning systems. 
Speeding Up Relational Reinforcement Learning through the Use of an Incremental First Order Decision Tree Learner.
Relational reinforcement learning (RRL) is a learning technique  that combines standard reinforcement learning with inductive logic  programming to enable the learning system to exploit structural knowledge  about the application domain.
A polynomial time computable metric between point sets.
Measuring the similarity or distance between two sets of points in a metric space is an important problem in machine learning and has also applications in other disciplines e.g. in computational geometry, philosophy of science, methods for updating or changing theories, . . . . Recently Eiter and Mannila have proposed a new measure which is computable in polynomial time. However, it is not a distance function in the mathematical sense because it does not satisfy the triangle inequality.
Learning a Go Heuristic with TILDE.

Executing Query Packs in ILP.
Inductive logic programming systems usually send large numbers  of queries to a database. The lattice structure from which these queries  are typically selected causes many of these queries to be highly similar. As a consequence, independent execution of all queries may involve a lot of redundant computation. We propose a mechanism for executing a hierarchically  structured set of queries (a &#034;query pack&#034;) through which a lot of redundancy in the computation is removed. We have incorporated our  query pack execution mechanism in the ILP systems Tilde and Warmr by implementing a new Prolog engine ilProlog which provides support  for pack execution at a lower level. Experimental results demonstrate  significant efficiency gains. Our query pack execution mechanism is very  general in nature and could be incorporated in most other ILP systems,  with similar efficiency improvements to be expected.  
Using Belief Networks to Neutralize Known Dependencies in Conceptual Clustering.

Using Belief Networks to Neutralize Known Dependencies in Conceptual Clustering.

Top%2Ddown induction of clustering trees
An approach to clustering is presented that adapts the basic top-down induction of decision trees method towards clustering. To this aim, it employs the principles of instance based learning. The resulting methodology is implemented in the TIC (Top down Induction of Clustering trees) system for first order clustering. The TIC system employs the first order logical decision tree representation of the inductive logic programming system Tilde. Various experiments with TIC are presented, in both propositional and relational domains.
Generalizing Refinement Operators to Learn Prenex Conjunctive Normal Forms.
Inductive Logic Programming considers almost exclusively universally quantified theories. To add expressiveness we should consider general prenex conjunctive normal forms which use existential variables. Learning with refinement operators is the most used learning method in ILP. To extend refinement operators to PCNF, we should first extend substitutions to PCNF. The usual substitutions often generalize a formula instead of specializing it because of the existential variables. In this article we define substitutions to specialize a given PCNF and a weakly complete downward refinement operator. Based on this operator, we have implemented a simple learning system PCL on some type of PCNF.
Instance Based Function Learning.
. The principles of instance based function learning are presented.  In IBFL one is given a set of positive examples of a functional  predicate. These examples are true ground facts that illustrate the input  output behaviour of the predicate. The purpose is then to predict  the output of the predicate given a new input. Further assumptions are  that there is no background theory and that the inputs and outputs  of the predicate consist of structured terms. IBFL is a novel technique  that addresses this problem and that combines ideas from instance based  learning, first order distances and analogical or case based reasoning. We  also argue that IBFL is especially useful when there is a need for handling  complex and deeply nested terms. Though we present the technique in  isolation, it might be more useful as a component of a larger system to  deal e.g. with the logic, language and learning challenge.  1 
Top%2DDown Induction of Clustering Trees.
An approach to clustering is presented that adapts the basic top-down induction of decision trees method towards clustering. To this aim, it employs the principles of instance based learning. The resulting methodology is implemented in the TIC (Top down Induction of Clustering trees) system for first order clustering. The TIC system employs the first order logical decision tree representation of the inductive logic programming system Tilde. Various experiments with TIC are presented, in both propositional and relational domains.
A Framework for Defining Distances Between First%2DOrder Logic Objects.
this paper we develop a framework for distances between clauses and distances between models. The framework can be parametrised by a measure for the distance between atoms. It takes into account subterms common to distinct atoms of a set of atoms in the measurement of the distance between sets. Moreover, for a constant number of variables, the complexity of the distance computation is polynomially bounded by the size of the objects. Initial experiments show that the framework can be the basis of good clustering algorithms.  The framework consists of three levels: At the first level one chooses a distance between atoms .  The second level upgrades this distance to a distance between sets of atoms. We propose a framework that is a generalisation of three polynomial time computable similarity measures proposed by Eiter and Mannila, and an instance which is a real distance function, computable in polynomial time. We develop also a binary prototype function for sets of points. Prototype fun
